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Abstract 

Eigenvectors and eigenvalues of discrete graph Laplacians are often used for manifold learning and 
nonlinear dimensionality reduction. It was previously proved by Belkin and Niyogi [3] that the eigen¬ 
vectors and eigenvalues of the graph Laplacian converge to the eigenfunctions and eigenvalues of the 
Laplace-Beltrami operator of the manifold in the limit of infinitely many data points sampled indepen¬ 
dently from the uniform distribution over the manifold. Recently, we introduced Point Integral method 
(PIM) [8] [15] to solve elliptic equations and corresponding eigenvalue problem on point clouds. We have 
established a unified framework to approximate the elliptic differential operators on point clouds. In this 
paper, we prove that the eigenvectors and eigenvalues obtained by PIM converge in the limit of infinitely 
many random samples independently from a distribution (not necessarily to be uniform distribution). 
Moreover, one estimate of the rate of the convergence is also given. 


1 Introduction 


The Laplace-Beltrami operator (LBO) is a fundamental object associated to Riemannian manifolds, which 
encodes all intrinsic geometry of the manifolds and has many desirable properties. It is also related to 
diffusion and heat equation on the manifold, and is connected to a large body of classical mathematics (see, 
e.g., [H]). In recent years, the Laplace-Beltrami operator has attracted much attention in many applied fields, 
including machine learning, data analysis, computer graphics and computer vision, and geometric modeling 
and processing. For instance, the eigensystem of the Laplace-Beltrami operator has been used for representing 
data in machine learning and data analysis for dimensionality reduction M, and for representing shapes 
in computer graphics and computer vision for the analysis of images and 3D models mm- 

In general, the underlying Riemannian manifold is unknown and often given by a set of sample points. 
Thus, in order to exploit the nice properties of the Laplace-Beltrami operator, it is necessary to derive In 
this paper, we assume that the data points, X = {x 1; • • • ,x„}, are sampled independently over the manifold 
A4 from a probability distribution p(x). On the sample points, we consider following discrete eigenvalue 
problem. 
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( 1 . 1 ) 


where R : R + —> R + is a kernel function , R(r) = / ( + °° R(s)ds. 

This eigenvalue problem is closely related with the eigenvalue problem of normalized graph Laplacian. 
The graph Laplacian is a discrete object associated to a graph, which reveals many properties of the graph as 
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does the Laplace-Beltrami operator to the manifold [5] . In the presence of no boundary and the sample points 
are uniformly distributed, Belkin and Niyogi [3] showed that the spectra of the normalized graph Laplacian 
converges to the spectra of Am . When there is a boundary, it was observed in 00 that the integral Laplace 
operator L t is dominated by the first order derivative and thus fails to be true Laplacian near the boundary. 
Recently, Singer and Wu [16] showed the spectral convergence in the presence of the Neumann boundary. In 
the previous approaches, the convergence analysis is based on the connection between the graph Laplacian 
and the heat operator. The analysis in this paper is very different from the previous ones. We consider this 
problem from the point of view of solving the Poisson equation on submanifolds, which opens up many tools 
in the numerical analysis for studying the graph Laplacian. 

The purpose of this paper is to study the behavior of discrete eigenvalue problem m at the limit of 
n —> oo and t —> 0. The main contribution of this paper is that our study reveals that when n —> oo and 
t —> 0, the spectral of m converge to the spectra of following eigenvalue problem. 

f div (p 2 ( x ) Vu ( x )) = M x ), xGTW, 

[ §%(x) = 0, xgOM. 

where n is the out normal vector of Ad . 

To analyze the convergence, we introduce an intermediate integral equation. 

lf M R ^ X ) (u( x ) - u(y))p(y) d y = R ( ^ X ) J(y)p(y)dy, xgAL ( 1 . 3 ) 

Similar integral equation also can be found in previous works. However, the rest of the analysis in this paper 
is very different as the previous ones. Before presenting the main results, we need to define three solution 
operators T,T t and T t ^ n - 


1.1 Solution operators 

The solution operators are defined as following. 

• T : L 2 (At) —*■ H 2 (M) is the solution operator of the problem (11.41) . i.e., for any / G L 2 (At), T(f) with 
Jm T(f) = 0 is the solution of the following problem: 

“?M div (P 2 ( x ) Vu W) = /( x )> 

tiM = o, 

where n is the out normal vector of At. 


x G At, 
x G d At. 


(1.4) 


T t : L 2 (M) L 2 (M) is the solution operator of following integral equation (11.51) . i.e. u = T t (f) with 

f M u(x)p(x)dx = 0 solves the following integral equation 


t?t(x, y)(u(x) — u(y))p(y)dy — / R t (x, y)f(y)p(y)dy. 


(1.5) 


>M 


> M 


where 


t?t(x,y) = 


1 


:R 


|x-y|| 

it 


(47 Tt) k / 2 

T ti „ : C(M) —> C(At) is defined as follows. 
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, Rt(x,y) = 
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= „ „„ „ M E R x j) u j + „ Wt n(x) E R ^ x j)/( x j) 
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( 1 . 6 ) 
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where w>t,ra(x) 
system, 


■i-^' l =1 Xj) and u = (iti,--- with Y^i=i u i = 0 solves following linear 


-j^ II IL 

— ^2 R t (xi,Xj)( u i - Uj ) = -^£ t (xi,Xj)/(xj) (1.7) 

3 =1 1=1 


To simplify the notations, we also introduce two operators. For any / £ L 2 (Af), 

it/( x ) =7 [ -R*(x,y)(/(x) - f(y))p(y)dy. (1.8) 

and for any / £ C(A4), 

1 " 

Lt,nf(-x) =—'52Rt(,x,Xj)(f(x.) - /( x l)). (1-9) 

J=1 

Using these definitions, we have that 

L t (T t f)(x) = ( R t (x, y)f(y)p(y)dy (1.10) 

J M 

and 
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Lt(T t . n f)(yLi) = -Y^Rt{xi,Xj)f(xj)- ( 1 - 11 ) 

n 1=1 

From (11.101) and (11.111) . we can see that in some sense, solution operators, T t ,T tjn , are inverse operators of 
L tjn , L t . So, it is natural to imagine that their spectra are equivalent. 

Proposition 1.1. Let 0(u) denote the restriction of function u to the sample points X = (xi, • • • , x n )*, i.e., 
Q{u) = (u(xi), • • • ,it(x„))‘. 

1. If a function u is an eigenfunction ofT tyn with eigenvalue A, then the vector 9(u) is an eigenvector of 
the eigenproblem ED with eigenvalue 1/A. 


2. If a vector u is an eigenvector of the eigenproblem m with the eigenvalue A, then I\ (u) is an 
eigenfunction ofTt, n with eigenvalue 1/A, where 


h(u)(x) 


Xt E"= 1 R t (X, Xj)Uj + E"=l Rt (X, Xj )Uj 

E"= 1 ^(x,x j ) 


3. All eigenvalues of T,Tt,Tt <n o,re real numbers. All generalized eigenvectors of T,Tt,Tt tU are eigenvec¬ 
tors. 


This proposition can be proved by following the same line as that in [Ill- 

Using this proposition, we only need to analyze the relation among the spectra of T and X/ n . In the 
analysis, the operator T t plays very important role which bridge T and T/„. The main advantage of using 
these solution operators instead of L t and L t , n is that they are compact operators which is proved in following 
proposition. 

Proposition 1.2. For any t > 0, n > 0, T,T t are compact operators on H 1 (Xi) into H 1 (A4); T tl T ti n are 
compact operators on C^Al) into C 1 (AI). 
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Proof. First, it is well known that T is compact operator. T t ,h is actually finite dimensional operator, so it 
is also compact. To show the compactness of T t , we need the following formula, 


T t u =—r^l - R t(x,y)T t w(y)dy H- [ R t {x,y)u(y)dy, Mu£H 1 {M). 

wtW Jm ™ t (x) Jm 

Using the assumption that R £ C 2 , direct calculation would gives that that T t u £ C 2 . This would imply 
the compactness of T t both in H 1 and C 1 . □ 

It is well known that compact operator has many good properties. Many powerful theorems in the 
spectral theory of compact operators can be used which makes our analysis concise and clear. 

1.2 Main result 

The main result in this paper is stated with the help of the Riesz spectral projection. Let X be a complex 
Banach space and L : X —> X be a compact linear operator. The resolvent set p{L) is given by the 
complex numbers z £ C such that z — L is bijective. The spectrum of L is cr(L) = C \p{L). It is well 
known that a(L) is a countable set with no limit points other than zero. All non-zero value s in er(L) are 
eigenvalues. If A is a nonzero eigenvalue of L , the ascent multiplicity a of A — L is the smallest integer such 
that ker(A — L) a = ker(A — L) a+1 . 

Given a closed smooth curve T C p(L) which encloses the eigenvalue A and no other elements of <r(L), 
the Riesz spectral projection associated with A is defined by 

E (\, L ) = ^-J^ z - L )- 1 dz, ( 1 - 12 ) 

where i = \f —T is the unit imaginary. The definition does not depend on the chosen of T. It is well known 
that E(A,L) : X —> X has following properties: 

1. E{ A, L) o E{ A, L) = E{ A, L), L o E{ A, L) = E{ A, L) o L, E( A, L) o E(p, L) = 0, if A ^ p. 

2. E( A, L)X = ker(A — L) a , where a is the ascent multiplicity of A — L. 

3. If T C p(L) encloses more eigenvalues Ai, ■ ■ ■ , A m , then 

E{ A 1} • • • , A m , L)X = ®Zi ker(Ar - L) a < 
where ai is the ascent multiplicity of A i — L. 

The properties (2) and (3) are of fundamental importance for the study of eigenvector approximation. 

To prove the convergence, we need some assumptions on the manifold M, probability distribution p(x) 
and the kernel function R which are summarized as following: 

Assumption 1. 

• Assumptions on the manifold: Ai is k-dimensional compact and C°° smooth manifold isometrically 
embedded in a Euclidean space R d . 

• Assumptions on the sample points: X = {x l5 • • • , x„} are sampled independently over the manifold Ai 
distribution p(x) £ C 1 (Ai ) and min xe jvip(x) > 0, max xe jv(p(x) < oo. 

• Assumptions on the kernel function R(r): 

(a) R £ C 2 (K + ); 
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(b) R(r) > 0 and R(r) = 0 for Vr > 1; 

(c) 3<5o > 0 so that R(r) > So for 0 < r < 

Now, we are ready to state the main theorem. Since T and T tjra are both compact operators, their 
eigenvalues can be sorted as 


()<■■■< A2 ^ Ai, 

0 < ■ • • < \\’ n <■■■< A2 ,n < A*’", 


where the same eigenvalue is repeated according to its multiplicity. 

For corresponding eigenvalues and eigenfunctions, we have following theorem. 

Theorem 1.1. Under the assumptions in Assumption QJ let A i be the ith largest eigenvalue of T (same 
eigenvalue is repeated according to its multiplicity) with multiplicity ai and 4^,k = l,--- , cc* be the linear 
independent eigenfunctions corresponding to A i. Let A*’" be the ith largest eigenvalue ofT tyn . With probability 
at least 1 — 1/n, there exists a constant C\ > 0, C 2 > 0 depend on Ai, kernel function R, distribution p and 
spectra ofT, such that 

,*!•■-A., <C,(^ + 1S2+J^i!±i 


and 


II J.k uv_t,» m ij.ku 1 4 . 1/2 , logn+|logt| + l\ 

II <Pi — R\ a i ! J-t,n)<Pi Ili^CM) < ^2 I t H ^fc+2 ^- )> 


as long as n large enough. Here o( n = {A f n £ <j(T t ^ n ) : j £ R} and R = {j £ N : A j = A^}. 


This theorem will be proved in Section 2 and 3. Some conclusions are made in Section 4. 


2 Proof of the main theorem (Theorem 11.11) 

The proof of Theorem 11.11 mainly consists of three parts. The first part is to relate the difference of the 
eigenvalues and eigenfunctions with the difference of operators T — T t and T t — T t}n fTheorem 12.41) . This is 
achieved by using one theorem in the perturbation of compact operators. 

To apply the theorem obtained in the first part, we need to estimate the difference of operators T — T t 
and T t — T t< n in H 1 and C 1 norm respectively. This is also the most difficult part. Comparing with the 
pointwise convergence which was proved in previous works, convergence in norm is much stronger and much 
more difficult to prove. Fortunately, under some mild assumption which are listed in Assumption [TJ we could 
prove that T t —> T in H 1 norm as t —> 0 (Theorem 1 2.5 11 and T ti „ —> T t in C 1 norm as n —> oo fTheorem l2.6l) . 

To get the rate of the convergence, in the last part of the analysis, we use the the theory of the Glivenko- 
Cantelli class in statistical learning to estimate the error in the Mote-Carlo integration. The key point in 
this part is to estimate the covering number of the function classes defined as following. 

Here, we list some notations which will be used in the proof. Some of them have been defined in previous 
sections. We also list them here for the convenience of readers. 

• k: dimension of the underlying manifold; d: dimension of the ambient Euclidean space; 

• C: positive constant independent on t and sample points X n . We abuse the notation to denote all the 
constants independent on t and sample points X n by C. It may be different in different places. 

• Ct = 2 is the normalize constant of kernel function R. 
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p(x): probability distribution function. 

R: kernel function. R(r) = / r °° R(s)ds. 

Rt{*-,y) = ji^mR > ^*(*.y) = jy^mR ( 

w t( x ) = Xm ^t(x,y)dy, w t , n (x) = ra(47r 1 t)fe/ 2 E"=i # (■ 

W iriin,Wmax: Wmin = inf min u> t (x), w max = supmaxw t 
t>0xSA1 ooxeM 

we can show that 0 < w m ini w) max < oo. 


Iix-y|| 2 \ 

4 * ) ■ 

l x " x b 2 N i 

4t ; 

(x). Under the assumption in Assumption [T] 


P(f) = I M /( x M x )dx, Pn(f ) = | E"=i /( x 0 - 


K. = {«(AA) 

{«(*#) -xcm} 


Rt = 


v t = 


:xeM} 

K t -^,n = {^R(^)R(^)-. x £M,z£M} 

^•^ = {^(¥)^(¥) :x 6^ z ^} 


A 

Uf 

Uf 


• (t) V z i? (^#) : x £ At, Z £ m } 

Kt,n = {^R (^#) v z i? (^#) :xeAi,zeM} 

Kt, n = {^R (^#) V ZJ R (J=#) :xe^zeAf} 


2.1 Perturbation results of solution operators 

First, we need some results regarding the perturbation of the compact operators. 

Theorem 2.1. (\Tj) Let ( X , || • ||x) be an arbitrary Banach space. Let S and T be compact linear operators 
on X into X. Let z £ p(T). Assume 


||(:r-s)S||*< 


Then z £ p{S) and (z — S) 1 has the bound 


\\(z-T)-'\\ x - 


( 2 . 1 ) 


||(*-S)- 1 ||x< 


i + ||5|UII(^-t)- 1 |U 
H-||(.~-T)-i|| x ||(T-5)S||x' 


( 2 . 2 ) 


Theorem 2.2. (JVj) Let (A', || • ||x) be an arbitrary Banach space. Let S and T be compact linear operators 
on X into X . Let zq £ C, zo ^ 0 and let e > 0 be less than \zo\, denote the circumference |z — zq\ = e by T 
and assume T C p{T). Denote the interior ofT by U. Let <tt = U D cr{T) ^ 0. as = UC\a(S). Let E(as , S) 
and E(ctt,T) be the corresponding spectral projections of S for as and T for aT, be. 


E(as,S) = ^- [(z-S^dz, E(a T ,T )=-L [(z-T^dz. 
Z'K'l J p Z'K'l Jy' 


(2.3) 
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Assume 


|| (T- S)S\\ X < min — 
zer || [z 


T)~ l \\x 


(2.4) 


Then, we have 

(1) . Dimension E(as, S)X = E(ax,T)X, thereby ag is nonempty and of the same multiplicity as ax- 

(2) . For every x £ X, 

II E(* t ,T)x - E(a s ,S)x\\x < — (||(T - S)x\\ x + Mx||(T - S)S \\ X ). 

Co 

where M = max zer ||(2 - T) _1 ||. Y , c 0 = min zgr |z|. 


Lemma 2.1. (fL$) Let T be the solution operator of the Neumann problem O and z S p{T), then 

IK Z — T)~ 1 \\h 1 ( M) < max 

where {A„} rag N is the set of eigenvalues of T. 

Lemma 2 . 2 . (\14V Let T t be the solution operator of the integral equation (USD- For any z & C\ UneN B (^n, r 0 ) 
with ro > ||T — T t W h 1 , then 

ll^-T^Hc^maxj^M^ (mm|^-A„|- ||T-T t || ff ^ , j|| | - 

Theorem 2.3. (\14V Let T t be the solution operator of the integral equation (O and X n be eigenvalues of 
T, then 

a(T t )c (J 5(A n ,2||T-T t || ff i (A1) ). 

nG N 


The main result in this subsection is stated as following in which the difference of the eigenvalues and 
eigenfunctions are related with the difference of the solutions operators. 


Theorem 2.4. Let X m be the mth largest eigenvalue ofT with multiplicity a m and k = 1, • • • , a m be the 
eigenfunctions corresponding to A m . Let A Iff be the mth largest eigenvalue ofT tjn . Let 7 m = min |Aj — Aj+i| 


II (Tt, 

||T- 


t 7m i fc/4+3/2 (|A m |-7 m /3) 2 t (fc+2)/4 7r, 


n Tt)Tt tn \\ cl <min{g 7 24 ’ 12 ’ 

TtWH'tM) < 7m/12, ||(r — T t )T t \\ H l(M) < (|Am| — 7m/3)7m/3 


(|Am| - 7m/3)% 
2 4 


Then there exists a constant C\,C 2 depend on Ai, the kernel function R, ”f m and X m , such that 
IA^ - A m | < ^J ^ ||(T t ,„ - r t )T t ,„|| cl + ||T - T t \\ m{M) 

and 

\\ ( / > m~E(o' i ff l j Tt, n )(t>m\\ H i( M ) < C{^{T—Tf)<f^\\ H i+\\[T—T t )T t \\ H i)+-^j^^^{\\{T t —T tr ^(j) , f n \\Qi-\-\\{T t —T tn )T t>n \\ci) 

Here a^ = {A‘ ,n € cr(T ti „) : j S I m } and I m = {j S N : A j = X m }. 
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Proof. Let n = ik/ ? +3/2 \\{T t>n - T t )T t , n \\ cl + \\T - T t \\ H i {M) , A = C\ [U„ eN -B(A„,n)|J-S(0,t 1 / 2 )], For any 
z € A, using Lemma l2~Tl we have 


^-Tt )- 1 He 1 < 


2\M\ 


< 


| a | t (fc+a)/4 

2\M\ 


min |z - X n \ - \\T - T t \\ H i 


-l 


fk/ 4+1 


(n- ||r-T t || ff i)“ 


t 1 / 2 


< 


or 


: - T t y 


IC 1 > 


< — < —— < 


||(r t , n -T t )T t , n || cl - \\(T t , n -T t )T t , n \\ cl 

Vi 


< 


z | " t 1 / 2 “ ||(T t , n -T t )T t> J cl - ||(T t , n -T t )r t , n || c 


Here, we use the condition that || (T ti „ — T , t )T’ tjTl || cl < i/2. 
Both above two inequalies implies that 




(z-T t )~i\\ c P 


Then using Lemma [2~il we have 0 £ p(T t , n )- 

Since z is arbitrary in A. we get A C p(T t , n ). This means that 

a(T t , n ) = C\p{T t>n ) C C\.A = (J B(X n , n ) |J B( 0, t 1 ' 2 )- 


Moreover, using Theorem 12.31 and the definition of r i, we have 

a(T t ) C (J H(A n ,2n). 


n(E N 


(2.5) 


( 2 . 6 ) 


For any fixed eigenvalue A m £ cr(T), let = min | A ? — A 7+ i |. Using the structure of er(T), we know that 

j<m 

7 m > 0. Since 


f k/A+3/2 ^)^,r l || C i ^ 7m/12, ||T || #1 (_vj) < 7 m /12, 

we know that ri < 7 m /6. 

Let Fj = (z £ C : |z — A 7 | = 7j/3}, Uj be the aera enclosed by Fj. Let 

a t,j = cr (^t) f^| Uj, &t,n,j = &{Tt,n) f"') Uj. 

Using the definition of we know for any j < m, Tj C p(T),p(Tt) and p(Tt, n )- 
In order to apply Theorem 12.21 we need to verify the conditions 


\\(T-T t )T t \\ H i < min 


\\(z-T)-i\\ h A 


\\(T t - Tt^T^nWc 1 < min 


\\(z-T t )-i\\ c U 

Using Lemma 12.11 and the choice of Fj, we have 

\ z I . min ze r m \z\ , /Q . . , . , 

mm 77 - — > -T77- Tn — > ( A m - 7m/ o) mm z - A m = 

jer m (z-T - 1 rnax 3e r m (2 - T - 1 'zer m ,neN' 


(2.7) 

( 2 . 8 ) 


(|A m | - 7m/3)7m/3- 















Then, using the assumption that ||(X 1 — T t )T t ^m(M) (|Am| — 7m/3)7m/3, condition (12.71) is true.. 

Using Lemma [2~?1 we have 


> 


mim e r m \z\ 


zer m ||(5r-r t )- 1 || C n max j£ r m ||(2 - Tt)- 1 ^! 

(|A m |-7m/3) 2 t( fe+2 )/ 4 


> 


> 


min \z-\m\ - \\T-T t \\„i 
z&r m ,n^N 


(|A m | - 7m/3) 2 t (fc+2)/4 7r, 
12 


or 


> 


mm z 6 r m \z\ 


zer m || (z-T t )- l \\ C i rnax, e r m ||(z - 


> 


(| A m | - 7m/3) 2 


(2.9) 


( 2 . 10 ) 


To get the last inequality of (12.91) . we use the assumption that ||T — T t ||#i < 7/6 and min \z — A m | = 

zer m ,neN 

7m/3. 

Using the assumption that ||(T— T ttn )T ttn \\c^(M) — min{ d A m|- 7 m/ 3 )^ A + 17 7 m ^ (|A m |- 7 m / 3 ) ^ condition 
is satisfied. 


( 2 . 11 ) 


Then using Theorem 12.21 we have 

dim(£(A m ,T)) = dim(E(<j t ,m, T t )) = dim(£’(cr tin , m , T ti „)). 

Using m , above equality would imply that 

I Am" ^ Am| < n = ||(T t ,„ - T t )T t . n\\ cl + \\T - T t \\ H i (M) . (2.12) 

The convergence of eigenspace is also given by Theorem 12.21 For any x £ E(X m ,T), ||x||ci = 1, 


||x - E(a tt m,T t )x ||jji < 


max z 6 r m ||(2 - T) 1 ||i/i 7 m /3 


mm ze r„ \z\ 


(\\(T-T t )x\\ H i+\\(T-T t )T t \\ H i\\x\\ H i). 


Using Lemma 12. 1[ we know that 


1 3 

max ||(2 — T)~ 1 \\ h i < max -— < -—, 

zer m jgn 1 2 Aj | 27 m 


and min ze r m | 2 | = |A m | — 7m/3- This implies that from Theorems 12.51 

\\x - E(a t , m ,T t )x\\ H i <C(\\(T-T t )x\\ H i+\\(T-T t )T t \\ H i\\x\\ Hl ). 
Regarding the convergence from T tjJ1 to T t , using Theorem 12.21 again, we have 

7 m max ||(2 - Tt)" 1 !^! 


(2.13) 


II E(<j t ,m, T t )x - E(a t: n,m, T t>n )x llc -1 < 
Using Lemma 12.21 we know that 


2 GT„ 


3 min ze r m |^| 


(II (T t - T t , n )x\\ c i + ||(It - T tin )T tin || c i). 

(2.14) 


max ||(2 - < max j jr^) 7 I 


(mm|a-A J |-||T-r,||„.) ,kj 


< max 


12 


. 7m(| Am| - 7m/3)t( fc + 2 >/ 4 ’ |A m | - 7m/3 J ' ^ 

To get the last inequality, we use that ||T—T t ||#i < 7 m /6 and |2 — A m | = 7m/3, | 2 | > |Am — 7/3| for 2 £ T m - 
Then the proof is completed by combining (12.121) . (12.131) . (12.14[) and (12.151) . 

□ 
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2.2 Convergence of solution operators 

To apply Theorem 12.41 we need to estimate the difference of the solution operators. More precisely, we need 
to estimate \\T — T t || ff i and \\T t — Tt >n ||cfi as t —> 0 and n —► oo. These results are summarized in Theorem 
12.51 and Theorem 12.61 respectively. 

Theorem 2.5. Under the assumptions in Assumption Ql there exists a constant C > 0 only depends 

on At and the kernel function R, such that 

\\T-T t \\ m KCt 1 ' 2 , ||T t ||tfi < C. 

The proof of this theorem can be found in [T3]. 

The other theorem is about \\T t — T t>n ||cu. 

Theorem 2.6. Under the assumptions in Assumption^ and 

C t sup \p(f) -Pn(f)\ < Wmin/2, (2.16) 

/ G7?. t /U7^.tU7^8t 

Ct sup \p(f)-Pn(f)\ < ~ -7----7> ( 2 - 17 ) 

f&lC t , tn uK t , n -ic t , n 2max{w max + u> m in/2, 2/w m i n \ 

where S = 4w w "f^ w , t' = i/18. There exists a constant C only depends on At and kernel function R, 
such that 

IIPU - T.)T«,„|| 0 . < ll(T.,„ - T,)f\\ c , < TAfL. 

where 

h = sup \p n {g) ~p(g)\ +t _ _sup _ \Pn(g)-p(g)\ 

gen t ICt,n'JKt g£V t UK.t,n-KtUK.t, n -KtUlCt,n-'D t 

+t 2 sup_ \p n (g) ~p(g)\ +t 3 £up_ \p n (g)-p(g)\, 

g€Klt,n‘'Dt pG/Ctjn'^i 

Hf) = sup \pn{g) - p{g)\ + 1 _ su P _ \p n {g)-p{g)\ 

g€KflCt, n UKt gG'DtUf1ZtUlCt,n-'R.tUlCt,n-'Dt 

+t 2 sup_ \p n {g)-p(g)\ +t 3 ^up_ \p n {g)-p(g)\, 

g£fc't,n''Dt 

The proof of this theorem will be deferred to Section [31 

2.3 Entropy bound 

In this subsection, we will verify the assumption (12.161) . (12.171) in Theorem 12.61 and estimate ho and h(f) 
defined in (12.181) and (12.191) to get the convergence rate. The method we use is to estimate the covering 
number of function classes defined in previous subsection. First we introduce the definition of covering 
number. 

Let (Y, d) be a metric space and set F C Y . For every e > 0, denote by N(e, F, d) the minimal number 
of open balls (with respect to the metric d) needed to cover F. That is, the minimal cardinality of the 
set {y±, • • ■ , y m } C Y with the property that every f £ F has is some y,; such that d(f,yi) < e. The set 
{?/i t ‘ ‘ ‘ iZ/m} is called an e-cover of F . The logarithm of the covering numbers is called the entropy of 
the set. For every sample {xi,-- - ,x n } let y n be the empirical measure supported on that sample. For 
1 < p < 00 and a function / , put \\f\\L p (n n ) = (£ \f(xi)\ p ) 1/p and set \\f\\oo = maxi<i< n |/(xj)|. Let 

N(e, F, L p (p n ) be the covering numbers of F at scale e with respect to the L p (y n ) norm. 

We will use following theorem which is well known in empirical process theory. 


(2.18) 

(2.19) 
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Theorem 2.7. (Theorem 2.3 in ms) Let F be a class of functions from Ai to [—1,1] and set p to be a 
probability measure on Ai. Let (xj )^ 1 be independent random variables distributed according to p. For every 
e > 0 and any n > 8 /e 2 , 

P ( sup |-V'z(xj) - f /(x)/z(x)dx| > e ] < 8 E M [AT(e/ 8 ,i r ,Li(/x n ))] exp(-ne 2 /128) (2.20) 

Notice that 

Ll(pn') — Loo(.Pn) Lqq 


where H/Hl^ = max x6 ^ |/(x)|. Then we get following corollary. 

Corollary 2.1. Let F be a class of functions from Ai to [—1,1] and set p to be a probability measure on Ad- 
Let (xj)^. 1 be independent random variables distributed according to p. For every e > 0 and any n > 8/e 2 , 

P ( sup |- V'Z(xj) - [ /(x)//(x)dx| > e ] < 8N(e/8,F,L 00 )exp(-ne 2 /128) (2.21) 

\f£F J M J 

where N(e,F,L oa ) be the covering numbers of F at scale e with respect to the Loo norm 

Corollary 2.2. Let F be a class of functions from Ai to [—1,1]. Let (xj )^ 1 be independent random variables 
distributed according to p, where p is the probability distribution in Assumption Q] Then with probability at 
least 1 — 5, 

lnN(J-,F,L 00 ) + In ^ 

V n o 

where 

r 1 " 

p{f) = / /(x)p(x)dx, p n (f) = - y'Z(xi). ( 2 . 22 ) 

n i=1 

Proof. Using Corollary 12. II with probability at least 1 — <5, 


SUP \p{f)~Pn{f) | < a 
f£F \ 



where e$ is determined by 


Obviously, 


which gives that 


Then, we have 


which proves the corollary. 


sup \p(f)-p n (f)\ < eg, 

feF 


es = 



In N(es/8, F, Loo) + In ■ 



Nieg/SjFjLoo) < N( 



es < 




i F, Lqo) + In — 


□ 
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Above corollary provides a tool to estimate the integral error on random samples. The key point is to 
obtain the estimates of the covering number. 

Let us start from the function class IZt- The functions in TZ t are bounded uniformly, and the bound 
only depends on the kernel function R. To apply above corollary, we need to normalize TZt to make it lie in 
[—1,1]. Here we also use TZt to denote the normalized function class and absorb the bound of IZt into the 
generic constant C. We do same normalize procedure for all function classes defined in Section [2] 

Since the kernel R G C 2 (A4 ) and M. G C °°, we have for any x, y G M 


I R 


(M) 


(M) 


c „ 

£ 7t x 


z||. 


This gives an easy bound of N(e,TZ t ,L 00 ), 


N(e 1 TZ tl L 00 ) < (JL 

Using Corollary 12.21 with probability at least 1 — l/(2n), 


sup | p(f) -Pn(f) I < -5= (In n- Inf + 1) 1/2 

f£n t un t ,un B t V n 


(2.23) 


(2.24) 


Then, we have 


Corollary 2.3. With probability at least 1 — l/(2n), 


Wmin 


sup \p{f)-Pn(f) \< 0 

/G7?-tU7?. t /U7?.8t " 

as long as n is large enough such that the right hand side of ([2.241) is less than w m \ n /2. 

To get the covering number N(e,lCt 1 n, Loo), we need the assumption that supj gTCt | p(f) — p n (f)\ < 


omm 

2 - 


Wt,n(y) . 


R 


l x ~ y||' 

At 


- R 


l 2 ~y|| s 

At 


<- 


W min 


-| 1 ? 


l*-y|| 

At 


-R 


l z ~y|| 

At 


C . 

^ |x - y| 


The first inequality comes from the fact that min zg _v( u>t jn (z) > u; m in/2 which is guaranteed by the assump¬ 
tion that supj gTCt | p(f) — Pn(f) | < Then we have 


/ (j 

N{e, lCt,n> Loo) A ( ^ 

Similarly, we can get 

N(e, lCt, n • K-t,m Lqo) < 
Using Corollary [521 if sup /gKt | p(f) - p n (f) \ < ^, then 


C 

e\ft 


2k 


(2.25) 


(2.26) 


sup \p(f)~Pn{f)\ < C\ -(lnn-\nt + l) 1/2 (2.27) 

feKt,nUKt, n -K t ,n * n 

with probability at least 1 — l/(2rc). From Corollary 12.31 we know that the assumption sup f^n t | p{f) — 
p n (f) | < 2 Af J1 holds with probability at least 1 — l/(2n). By integrating these results together, we obtain 
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Corollary 2.4. With probability at least 1 — 1 /n, 


sup \p{f) -p n (f)\ < 


_ P _ 

2 max{ir max T W m i n /2, 2/lC m i n } 


as long as n is large enough. Here <5 = iw • 

Using similar techniques, we can get the estimate of h 0 and h(f) in (12.181) and (12.191) . Together with 
Theorem 12.51 we get 

Theorem 2.8. Let <f> be an eigenfunction of T. With probability at least 1 — 1 /n, 

\\(T t - Tt^Tt'uWc 1 ^f3k/4+3/2~f^ (l nn - hit + 1) ^ , 

IIm <>”" - >"* + D 1 ' 2 

as long as n is large enough. Here C<t, is a constant depends on Ai, kernel function R, distribution p and 
eigenfunction <j>. 


3 Proof of Theorem 12.6 

To prove Theorem 12.61 we need following two theorems. 

Theorem 3.1. Under the assumption in Assumption^ and assume (12.161) . (12.171) hold. There exist constants 
C > 0 only depends on Ai and kernel function R, so that for any u = (iti, • • • , u n Y £ with Ui = 0, 

n „ n 

J2 Rt(xi,*]){ui-uj ) 2 >-J2 u2 i- O 1 ) 


The proof of this theorem can be found in Appendix. 

Theorem 3.2. Suppose u = (ui,-- - ,«„)* with Y^i u i = 0 solves the problem (11.71) and f £ C(A4). Then 
there exists a constant C > 0 only depends on A4 and kernel function R, such that 

1/2 

<< 711 / 1100 , 

as long as (12.16[) . (12.171) are satisfied. 

Proof. Since (u i, • • • ,u n ) satisfies that 

^ n i n 

3 =i i =1 



1 " 

-E/( x *) 2 


i= 1 
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using Theorem 1 3. 11 we have 


C n 1 n 2 n 

— Rt{^-x.j){ui - Uj) 


i,j =1 


»>J=1 


2 

=-2 5Z ^t( x i, x i)/( x j> 


*>j=i 


1/2 


1/2 


< Z2 H i? t ( x i, x /)/ 2 ( x /) -2 5Z Rt{Xi,Xj)l 




*ij=l 


1/2 


1/2 


=^£/ 2 M 


J = 1 


2 — 1 


<c||/|U 


1/2 


□ 

Theorem 3.3. Under the assumptions in Assumption ]! J assume u(x) solves the following equation 

- L t u = r, (3.2) 


where 


LtU = ^t J M r ( ^ 4^ ) ~ u (y))p(y) d y- 

Then, there exist constants C > 0,To > 0 independent on t, such that 

\W\\l*(m) < C\\ r \\ L 2 ( M )- 


(3.3) 


(3.4) 


os long as t < To. 

The proof of above theorem can be found in m- 

Theorem 3.4. Under the assumptions in Assumption QJ Let f € C(A4) in both problems, then there exists 
constants C > 0, so that 


\\(Tt, n -Tt)T ttn f\\ L 2 ( M) < -T_-||/||oo ( sup \p n (g) -p(g)\ +1 sup \pn(g)-p(g)\ 

1 ' \gen t uK t -ic t , n g&lc t , n -n t uK t , n -n t ) 

||(T t , n -T t )f\\ L2(M) < -fc^rill/lloo ( SU P \Pn(g) ~ p(g)\ +1 sup \p n {g)-p{g)\ ) , 
t/+ \gen t UK t K t , n ff G/C t ,„TC t U/-TC t ) 


prll/lloc ( 

V 

as long as t small enough and (I2.16H . (12.171) are satisfied. 


Proof, of Theorem \3.f\ 
First, denote 


Ut,n(x) = T tt nf 


1 

nw ti n(x) 


Y R t(x,Xj)Uj 

3 =1 




(3.5) 
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where u = (ui, ■ ■ ■ , u n )* with ]T" =1 u i = 0 solves the problem (11.711 . fj = f(xj) and u> ti „(x) = i y =1 .Rt( x , x j')- 
And denote 


^i,n( x ) — Tt^ n Ut 7 n — 


where v = (v±, ■ ■ ■ , v n Y with ^" =1 Vi = 0 solves 


l ( n n \ 

I £*•(*,*>, I 


(3.6) 


^ Tl 71 

- ~ t '52M*i,Xj)( v i- v j) = -J2Rt(xi,Xj)uj- 
1=1 1 = 1 

It follows from Theorem 13.11 that there exists a constant C > 0 independent on t and n such that 


(3.7) 


1 


1/2 


1 


1/2 


1 


1/2 


£«n <cii/iu i -£«? <c -£«n <cii/n 


(3.8) 


i=l 


i=l 


i=l 


The idea to prove the theorem is using Theorem 13.31 Then we need to estimate || L t {Tt,n — T t )T t ,nf ||2 and 
|| L t (T t>n - T t )f || 2 for any / G C(M). 

For any / £ C(M), 


Lt{T tt n — Tt)Tt,nf — (L t T ti nTt,nf ~ ^t,nTt,nTt,nf^) + (L t} nTt,nTt,nf ~ L t T t Tt, n f) 

— (LtVt,n Lt^n^t^n) T (I J t,nTt 7 n'Ut,n LtTtUt^n) • (3.9) 

Next, we estimate two terms of right hand side of (13.91) separately. For convenience, we split n = 
a t , n + b t , n and 


at,n( x ) = 


n W t n (x) 

’ .7 = 1 




bt,n ( x ) =-r^y'i?t(x,Xj)Uj. 

For ||LA,n - Z^rA^lh, we have 

| (-^£^,71 Lt^bt^n) (x) | 

1 r ^ 71 ^ 

= 7 / i? t ( x ,y)(6 t ,„(x) - 6t,„(y))p(y)dy-V i?t(x,Xj)(6 tjn (x) - b t ,„(xj)) 

t Jm n “7 


(3.10) 

(3.11) 


i=i 


< -|^t,n( x )| 


IM 


Rt (X, y)p(y)dy-V (x, x.,-) 

n ^^ 


i=i 


l 

+ ~t 


f 1 71 A 

/ #t(x, y)6t, n (y)p(y)dy - - E i? t (x, Xj)& tj „(Xj) 

JM n j =1 


(3.12) 


The first term of (13.121) can be bounded as following, 


bt,n (x) / i?t (x, y)p(y)dy-E ( x , x j) 


/iw 


i=i 


L 2 


< Ctll^.nllz, 2 sup \p n {g)-p{g )| (3.13) 

gZi'R't 
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and 


\\bt,n\\li = -~ 


n2 Jm \ Wt,n(*-) 


Ct 


p(x)dx 


3=1 


< — / -X^( X ’ X J') H^( X ’ X 7) U .7 p( x ) dx 


) M 

2 ™ 

< —V 

n ' 


3 = 1 


V3 = l 


Ci 


< 


3 = 1 

Ct 2 — 


u 2 / i?t(x, Xj)p(x)dx 


3Vf 


£»i<« 2 


3 = 1 


where last inequality comes from (13.81) . 
For the second term of (13.121) . 


f 1 n 

/ #t(x,y)& t ,„(y)p(y)dy-V i? t (x, x J )6 t , n (x i ) 

•Am n J=1 


i? t (x,y) / ^(y, Xfc K]p( y )dy-^^^ E R t {^^ k )u k 


Im W t,n{ y) y 


x fe eP 


n -H ™t,n(Xj) 


x fe eP 


< -Z)KI 


fc=l 


Rt(x-,y) 5 u if R t (x,xj) 3 , . 

-i? t (y,x fc )p(y)dy-2^ “— 7^T 


Im wt,n(y) 




Let 


A = C t 


C t 


1 


R 


Im Wt,n(y) V 4i 


|x-yp 


|x* - y| 2 


At 


p(y)dy 


E 


i 


-R 


71 ^ W tA X j) \ 4t 


At 


We have 


\A\<C t sup_ \p n (g)-p(g)\ 

g£fct,n''R't 

for some constant C independent of t. In addition, notice that only when |x — Xj| 2 < 16t is A ^ 0, 
implies 

\A\ - 


32 1 


Using these properties of A, we obtain 

[ 1 , 

/ R t (x,y)b tt n(y)p(y)dy -V R t (x., x J )& t , n (x J ) 

Jm n 

L n 

■l-^loo E \ U k\ R 


3 = 1 


< 


Ct, ,, /|x-x fc | 2 


fe=l 


Ct 


< — y' c t \uk\R 

n z ' 


fc=l 


32t 
2 


|X - Xfc 

32t 


Ct sup_ |p„(c/)-p(fif)| 

pG/Ct,n,-7?.t 


(3.14) 

(3.15) 

(3.16) 

(3.17) 
which 

(3.18) 

(3.19) 
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It follows that 


f 1 71 ' 

/ #t(x,y)& t , n (y)z>(y)dy-^ R t (x, x j )6 tin (x J -) 

J M n A 1 


< ct 


3=1 


1 „ /|x-x fc | 2 


/X 


n 


- Vc t K|i? 

r) z —' 
fc=l 

1/2 


32f 


1/2 

p(x)dx | C t sup_ |p„( 5 ) -p(ff)| 


/ n \ ^ 

- _ E u fc Ct sup _ “P(sOI 

\fc=l / g&ICt.n-Ht 

< CtWfWooCt sup_ K(5)-p(ff)| 

9€fct,n‘'R't 


(3.20) 


To get the second inequality, we use the condtion that Ct sup ff67 j gt < w m i n /2. 

Now we have complete upper bound of ||L t & t>n — Tt,n&t,n|U 2 using (13.121) . (13.131) and (13.201) and C t = 


(47T t) k / 2 


c 


sup _ \p n {g) -p(g)\ 


II Ltbt^n -^t,n^t,n\\L 2 (A4) — ^^H^l 

Mimicing the derivation of (|3.21[) . we have 

\\L t a t ,n - Lt,nat,n\\l?(M.) < ,S 2+1 ll/lloo ( SUp bn(ff) -p(s)l 

t‘' + ygeKtUKt^-Kt J 


And consequently, 


||TtUt )7 x Lt^ n Vt^n || L 2 (A4) 

A Lt^n^t.n || L 2 (A4) T Lt ,nbt ,n || L 2 (A4) 

c „ „„ ( 


< 


fk/ 2+1 I 


sup bn(5) -p(5)| +i sup bn(ff) -p(ff)| 


k g€7Z.tU)Ct,n‘'R't 


g£fct,n‘'R't 


The second term of (13.91) can be bounded as following, 

CtiTtUt^n) Ct^ n (Tt^ n Ut^n) 

r i n 

< / ^t(x,y)u t>n (y)p(y)dy-V R t (x, Xj)uj 

J M. ^ i 


J = 1 


1 ^ R t (x,Xj) 




n 2 ^ ^t.n(Xj) 
J_ f -Rt(x,y) 

nJ M Wt,n(y) 


^-Rt(Xj,X fc )u fc - t^i? t (Xj,Xfc)/fc 


\,fc=l 


fc=l 


^i? t (y,x fc )u fc -t^2Rt(y,x k )f k J p(y)dy 


V/c=l 


fc=l 


= ± E-M ± E 


fc=l 


1 R t (x,Xj) 

r( Wt,n( X j) 


Rt(xj,x k )~ [ j^ t (y, X fc)p(y)dy 

Jm Wt,„(y) 




--E/^EE 


1 ^ i?t(x, Xj )- 


fc=l 


~( Wi,n(Xj) 


Rt(xj,x k )~ j 
J A 


R t ( x i y)._R t (y,xj.)p(y)dy 


VW ^t,n(y) 


) 


(3.21) 


(3.22) 


(3.23) 


(3.24) 
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Using the similar derivation from (13.1511 to (13.211) . we get 


II Lf (T t u t>n ) Lt^ n {Tt n Ut,n)\\L 2 

l n \ V2 

- C W 1 I Ct sup_ \p n (g)-p{g)\ +Ct\\f\\ 00 C t j>up_ \p n (g) - p(g)\ 

V ^ j =1 / 9€fct,n’'R't 


< 


-£-| 


sup_ |p n (g) -p(g )I + i _sup_ b«(s) — JP(^)I • 

pG/Ct,n ■'R't ) 


The complete estimate follows from Equation (13.231) and (13.241) . 


C 


| Lt(Tt,n Tt)Tt,nf ||l 2 (.M) < 


SUP |Pn(5)-p(fl)| 


i <7 G7 ?.t U 7Zt ■ /Ct, ri 


+t sup_ \p n {g) -p(g)\ +t 2 _sup_ \p n (g) -p(g)\ 

g€K.t,n m 'R't g€.fct,n‘'R't > 


Similarly, we can also get 


C 


\Lt{Tt, n — T t )f)\\ L 2( M ' ) < tk / 2+ 


sup \Pn(g)-p{g)\ 


+t sup_ \p n (g) -p{g)\ +t 2 sup_ \pn(g)-p(g)\ ■ 

g£fct,n’'R't gzf-Kt J 


The theorem is proved by using Theorem 13.31 and above two estimates (13.261) , (13.271) 


Theorem 3.5. Under the assumption in Assumption^ and assume (12.161) . (12.171) hold. Then, 
constants C > 0 only depends on A4 and kernel function R, such that for any f G C(M), 

\\T t ,nf\\oo < «“ fe/4 |l/||oo, ||T t , n /|| i3 < C'll/lloo- 


Proof. From the definition of T t , n) we have for any / £ C(M) 


T t , n f = 


C t 


nw t ,n{x) 


Y, r 

i= 1 


At 


tc t 

Ui + -7T > R 

nwt,n(x) frf 


At 


/( x 0 


where (ui, • • • , u n ) satishes the equation 


Ct 

nt 


J2 r 


j=i 


At 


C t 


it uj) — y^-R 


1=1 


4t 


/( x b- 


Using Theorem 13.11 it is easy to get that 


1/2 


-E“! 


where C > 0 is a constant only depends on A4 and kernel function R. 


(3.25) 


(3.26) 


(3.27) 

□ 

there exist 
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Then 


\Tt,nf | < 


C t 


< 


< 


and 


nw t ,n 

w- 

tc t 

nw t ,r 

t( x ) 

Ct 


nw t ,n 

w- 

2 C t ' 

l 1 ' 2 

«Wn, 

) 

,nf\\b 

<2 


J2 r 

1=1 

n 

J2 r 

i= 1 
n 

E« 


c, 


4/ 


nw t , n ( x ) i 


J2 R 


i=l 


4/ 


1/2 


4/ 


i=l 


4/ 


■E' 

i=l 


1/2 


\ 1/2 

«?) + *ll/l|c 


t||/||oc < C'll/lloc- 


Ct 


E* 


Im nw ttn (x) 

< 4 ^X >^ 2 


4/ 


< C'H/H 


it?p(x)dx + 2/ 


2 II j*|| 2 


□ 


Now, we are ready to prove Theorem 12.61 The main idea is to lift the covergence from L 2 to C 1 by using 
the regularity of the kernel function. The details are given as following. 


Proof, of Theorem \2.6\ 

For any / £ C 1 (A4), let u t , n = T t n f and Vi = T t n u t}n fx.i), i = 1, • • • , n. Using the definition of T t and 
Tt,m T t ut, n and T t n u t} n have following representations 


T t u t ,n = —E f Rt(x,y)T t u t ,n(y)p(y)dy H-E f -R(x,y)u t , n (y)p(y)dy, 

w t ( x ) Jm w*( x ) Jm 

1 n t n - 

Ttn'Ut.n — 7 7" ^ /^t ( x ; x 7)llt T 7 T ^ /^( x ; x t)llj. 


itO t ,n(x) 


i=l 


nt»t,„( x ) “ 


(3.28) 


where Ui = ut, n {xi)> * = 1, • ■ ■ , n. We know that (ui, • • • , «„) and (iq, • • • , u n ) satisfy following equations 
respectively 


1 


n i n 

^^?t( x i, x i)(Wi - «/) = -^i?t( x i, x /)/( x i), 


1=1 


i=l 


^ n i n 

E^t(Xj,Xj)(lli - Wj) = -E- Rt (XiiXjK'- 




1=1 


Using Theorem 13.21 we have 


1/2 


1/2 


T>n s c wi». UE 


/1 " \ 1/2 

< C [- 5 > 2 J 


(3.29) 


Denote 


T t u t,n = 


T t U t,n = 


1 


Wi,n( x ) J M 

1 f 


Rt(x,y)Ttu ttn (y)p(y)dy 


t 


wt, n {x) Jm 


Rt(x, y)Tt,nU t , n (y)p(y)dy + 


Wt,n( x ) Jm 

t I 


Wlt,n( x ) 


-R(x,y)u t ,n(y)p(y)dy, 

R(x,y)u t , n (y)p{y)dy. 


> M 
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We will prove the theorem by upper bound T t Ut, n — T^ut^n, Tt u t,n — T 2 Ut, n and T 2 ut >n — T t ^ n ut,n separately. 
First, let us see T t ut,n ~ Tt u t,n- 


< 


< 


TtUt,n If Uf..n 

1 1 


W t ,n{x) W t (x) 

2 C t 


IM 


Rt{x,y)T t ut, n (y)p{y)dy 


+ t 


R(x,y)u t , n {y)p(y)dy 


' M 


W min 

c 


sup (\p n (g) - p(g)\) 


IM 


Rt(x, y)T t u ttn (y)p(y)dy 


+ t 


R(x,y)u t , n {y)p(y)dy 


IM 


-TskJI (\\ TtUt ,n\\ L2 + t\\u t , n \\ L *) SUp (| p n (g) ~ p(ff)l) 

1 ge TZ t 


C 


-TSUI IK«IU 2 SUp i\p n (g) ~ P{ 9 )\) 

t ' pG 'll* 


c 


— 73fe74 ll/llo° SU P i\Pn(g) -p(g)\), 

t g6 Tit 


Similarly, we have 


C 


|v( T t u t ,n - T}ut,n )| < (3fc+2) /A \\f\\oo sup (\p n {g) -p(g )|), 

+ " g&nt UVt 


C 


which proves that 

\\T t Ut in -T?ut,n\\ cl < , (3fc T 2)/4 ll/lloo sup (\p n (g) - p{g)\). 

' ge'R-tU'Dt 

Secondly, using Theorem 13.41 we have 


Tlu t , n ~ T 2 u ttU 

1 


W t ,n(,x) J M 

< Ct~ k / 4 \\T t Ut, n ~ T t} nUt,n\\ L 2 
= cr fc / 4 ||(T t -T tjri )T t , n /|| i2 
c 


Rt(x, y) (T t u t , n { y) - T tin u tin (y))p(y)dy 


(3.30) 


< 


£3fc/4+l 


sup \p n (g)-p(g)\ +t sup_ \p n {g)-p(g)\ +t 2 _sup_ \p n (g)-p(g)\ • 
gen t unfK t ,n g&ic t , n -n t geK t ,„-n t 1 


and 


| V (TfU t ,n - T?U t ,n) | 


W t ,n(x) J M 

< Ct- k ' 4+1 ' 2 II T t ut, n - T t , n u tt „\\ L2 
= Ct- fe / 4 + 1 / 2 ||(T t -T tin )T t;n /|| i2 

c . ( 


Rt(x, y) (T t u t} n(y) - T t}n u t ,n(y)) p(y)dy ) 


< 


fk/A+3/2 I 


sup \p n (g) - p(g)\ + 1 sup \p n (g) -p(g)\ + t sup \p n {g)-p{g)\ • 


ypG U7?-t ■ t , i 




g£fc't,n‘'R't 


This implies that 
\\T t Ut,n -If 

c ,, / 


< 


^fc/4+3/2 I 


(3.31) 

sup bn(ff) -p(s)| +t sup \ P n{g) - p(g)\ + t 2 sup \p n (g) - p{g )I . 




pG/Ct,n-7^t 


pG/Ct,n-7^-t 
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Now, we turn to estimate T t ^ n Ut, n — T^ut <n - Using (13.281) . we have 

1 (1 


Tt : n'U j t,n R t Ut^n — 


w t ,n{x) \ n z -' 
’ v 7 \ 2=1 


Vi? t (x, Xi K- / i?t(x,y)T t , n u tjn (y)p(y)dy ] 


t / 1 


tWt,n(x) 

Using (13.281) again, the first term becomes 


- Vi?( x , x i)wi - / i?( x ,y)«t,n(y)p(y)dy ] . 


1 " r 

~y2 R t{x,Xi)vi- / i?t( x ,y)Tt,„u ti „(y)p(y)dy 


< 


1 n ( 


- / ftt(x,y) 


/x 

1 


nW t n V x i J , 

’ J = 1 


E «• (**• x j) «>+E «• (*• - * 


3 =1 


nw t ,n(y) 


E Rt (y, + , , Y Rt &- X J‘) <U p(y)dy 


< 


1 I 1 ^t(x, x i) [ 

E E M u E (*> x to - 


nw t , n (y ) 
Rt(x,y) 


) 




3=1 


“ Wt,n(Xi) 
2=1 ’ 


/X 


Wt,n(y) 


(y,Xj)p(y)dy 


£ i R t( x , x i) 5 / ^ I R t( x ,y) ^ / \ t \a\ 

_ L u u _ L —rr^ 4 ( x *> x j)- / —rr ^ 4 y- x j py dy 

” ' w tA x i) Jm w t,n{ y) / 


j=i 


Using the similar derivation from (13.151) to (I3.21|) . we can get 
1 . ra „ f 

~Y R t(x,Xi)vi- / i?t( x ,y)T t , n M tin (y)p(y)dy 
n .//u 


j=i 


<7 / 1 


1/2 


1/2 


C / 1 


^ 7fc74 ( - E ^ J u p„ |p*to) - Pto)l + I - E 


< 


t fc /4 

£3fc/4 I 


J=1 


g£fc't,n‘ r R't 


3 =1 


Ct sup_ \p n (g) - p(g)\ 

g£fct,n''R't 


sup |Pnto)-Pto)l+* sup_ |p„to) -pto)l 

i<?e/C t ,„-7?. t g&K. t , n -Kt J 


The second term can be bounded similarly, 

1 " r 

- Vi?( x , x i)ui - / i?(x,y)Mi,n(y)p(y)dy 

Jm 

C (1 


1/2 


1/2 


< 


< 


t k/4 

c 


■E’ 

i=i 


C7 / 1 x ^ 

c t sup^ |p„to) -pto)l + y.fc/4—i ( - E A 2 ) Ct 1 u p_ to) -pto)l 


gelCt,n-Kt 


‘'R.t 


f3k/4 11J 

Now, we have 


sup_ tonto)^Pto)l+ i £ U P_ to*to)-pto)l 

p€/Ct,n’^t > 


(3.32) 


c / 

|Ut jn M tjn - T t 2 n t;n | < -g^Tjn/iioo sup tonto) - pto)l + i su p_ tonto) - pto)l + f2 £ up _ to^to) -pto)l 

1 \g£lCt, n -Kt gGKt.n-Kt geK. t , n K t 
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Using the similar method, we can get 


\^(Tt,nU t , n -T^u tyn )\ < ,3,5L 1/2 ||/||oo ( sup \p n (g ) - p(g ) I + t sup \p n (g) - p(g)\ + t 2 sup \p n {g) - p(g)\ 

* ' ' \geic t , n --D t geic t , n .v t gelc t , n -v t 

The estimate of \\(T t — Tt^Tqnllc 1 in Theorem 12.61 is proved. 

Similarly, we can obtain the estimate of || (T t — T tt „)f\\ci for any / £ C(Ai) which complete the proof. 

□ 


4 Conclusions 

In this paper, we proved that the spectra of the normalized graph laplacian m will converge to the spectral 
of a weighted Laplace-Beltrami operator with Neumann boundary condition m as t —> 0 and the number 
of sample points goes to infinity. The samples points are assumed to be drawn on a smooth manifold 
according to some probability distribution p. Moreover, we also give an estimate of the convergence rate. 
Up to our knowledge, this is the first result about the spectra convergence rate of graph laplacian. However, 
the estimate of the convergence rate in this paper is far from optimal. There are mainly two places in the 
analysis which can be improved in the future. The first one is the estimate of the integral equation m- 
Now, we only get L 2 estimate, however, in the spectra convergence analysis, we need C 1 estimate. In this 
paper, the regularity is lifted by using the regularity of the kernel function. The trade off is that a large 
number t~ k / 4 emerge which reduce the rate of convergence. The other place is the estimate of the covering 
number. The estimate of the covering number is very rough in this paper. More delicate method would give 
better estimate which could help to improve the estimate of the convergence rate. 


Appendix A: Proof of Theorem 13.11 

Proposition A.l. (\Wj) Assume both Ai and dAi are C 2 smooth. There are constants w min > 0,w m ax < 
+oo and Tq > 0 depending only on the geometry of At, so that 


Wmin < w t {x) = / f? t (x, y)dy < u; max 
JM 

as long as t < To- 

We have the following lemma about the function w^n- 

Lemma A.l. Under the assumptions in Assumption ]] 1 if Ct sup | p{f) — p n (f)\ < w m i n /2, 

fen t 

W m in/2 < W ti „(x) < W max + W m in/2. 


This lemma is a direct consequence of Proposition IA.1I and the fact that 


Wt,n(x) ~ C t 



(^) 


p(y)dy 


< C t sup | p(f) 

fen t 


Pn(f) |- 


Lemma A.2. Hi 51 1 13] ) For any function u £ L 2 (Ai), there exists a constant C > 0 only depends on At, 
such that 




-Rt(x,y)(u(x) 


■w(y)) 2 p(x)p(y)dxdy >C |u(x) - u| 2 p(x)dx, 

JM 


(A.l) 
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where 


u= u(x)p(x)dx. 
JM 


Now, we can prove Theorem 13.II 


Proof, of Theorem \3.1\ 

First, we introduce a smooth function u that approximates u at the samples X n . 


i(x) = 


C, 


T, R 


nw t ' n (x) “ V 4i ' 

’ v 7 1=1 x 


(A.2) 


where uv,„(x) = & R ( |x 4 ^‘ l ) and t' = t/ 18. 
Then, we have 

[ [ Rf (x, y) (u(x) - w(y)) 2 p(x)p(y)dxdy 

JM JM 


[ [ R f (x, y) 


^ n i n 

Y,Rt'(x,*i)ui-- -rr^^fe.yK I p( x My)dxdy 


nw t -„(x) 7 —' ’ nw t ',n( y) . , 


< 


[ [ y) ( —j-— V i? t /(x,x i )i? t /(x j ,y)(u i -Uj) ) p(x)p(y)dxdy 

\n wt', n {x-)wt',n{y) I 

[ [ Rt' (x, y)—- Rt>{x.,Xi)Rt'{x. j ,y){u i - Uj) 2 p(x)p(y)dxdy 

JmJm n z wt',„{x.)w t ’ i n(y) frf. 

-Rf'(x,x i )i2 t /(x J -,y)i? t -(x,y)p(x)p(y)dxdy ) (m-Uj) 2 . 


y . , , 

^ V-Dn JM Wt',n{X-)Wt', n(y) 


(A.3) 


Denote 


A = 


:-Rt' (x, {xj,y)Rf (x, y)p(x)p(y)dxdy 


Jm Jm 'Wt',n(x)w t ' : n(y) J 

and then notice only when |xj — Xj| 2 < 36t' is A ^ 0. For |x, : — Xj | 2 < 36t', we have 


A < 


< 


< 


/X JM 

CCt 


Rt' (x, Xj)i? t ' (x 3 -, y)i? t / (x, y)i? 

/ / R t '{x,Xi)R t '(x.j,y)R 

3 0 JjVf jm 


72f' 


-l 


i? 


72£ / 


P(x)p(y)dxdy 


CC* f f Rt ’( x , Xi)i? t /(xj,y)i? 

JM JJK 

(^1). 


72t' 
|x» -Xj| 
72t' 


p(x)p(y)dxdy 


p(x)p(y)dxdy 


< CC t i? 


Combining Equation (1A.3I) . (IA.4I) and Lemma [A. 2 1 we obtain 


CC t 

n 2 t 


it, R ) ( Ui ~ Uj ^ ~ Sm^ ~ w )^( x ) dx 


(A.4) 


(A.5) 


We now lower bound the RHS of the above equation using T £A =1 u i- 


u(x)p(x)dx 




-T, 

n ' 
o =i 


C t ( |x - x,- 
-rt 


M W t ' ,n (x) V 4i ' 


p(x)dj 


(A.6) 
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Notice that 


C t 


-R 


Thus we have 


u < 


< 


Denote 


i=i 


< 


< 


v 4/' ; 


c t 

^ Wf 

/ 

u(Xi) 

*( 


c t 


n Wt' t n(Xi) 

i=i ’ v 7 


4 1' 


<C t SUp \p(f)~Pn(f) |- 


i? 


4i' 


+ M sup l p (/)^ p «(/)l 


j=i 


/e/C 


+ sup l p (/)^ p «(/)l 


/=i 


/e/c 


— y 

n 2 ^ W t ', n { x i 
s,J=l 


i? 


4/' 


( U J Uj) 


1/2 


+ -^“3 SUP 


/=1 


/e/c t . 

1/2 


Cl 


^min \ W 


\ 1/2 / n \ 1/2 

■ fl( 1X * 4 / jl ) /g S £P b(/)-Pn(/)|, 


(A.7) 


A = 


C* 




R 


At' 


l x - x /l 

4 f 


n w i'A x j) 


V J -*■! I 

4t' 


p(x)dx — 


l x j ~ X /P 

At' 


and then |A| < C t sup | p(f) — p n {f) |- At the same time, notice that only when |x* — x/| 2 < 16t' is 

/e K t',n’ K t',n 

A ^ 0. Thus we have 

1/11 s ^'■ 4 ' R ( t w £ >- 

Then 


'M 


J (x)dx — — y m 2 ( x j) 


3=1 


< 


y |C t UjUi||A| 


i,Z=l 


< -4 sup b(/)-Pn(/)iy 

" fZKf.nK*. n iPl 




c t 

< -4 

n feK 


n / 

sup b(/) ^ Pn(/)| y C (J R ■ 


|X, - X/1 

72 1' 


X» ~ X; | 

72t' 


UiUl 


< («'max+Wmin/2)Ct SUp b(/) ~ Pn(/)| ( “ y 

/e/c t ,,„-/c t ,,„ \ n “7 


u 2 


(A.8) 


In the last inequality, we use the condition that Ct sup^ e7it | p(f) — Pn{f)\ < Wmin/2. 
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Now combining Equation (IA.5I) . (IA.7I) and (IA. 8 I) . we have for small t 


-VY^X;) = / M 2 (x)p(x)dx + (w max + W m in/2)Ct SUp b(/) ~ Vn (/) | ( ~ V' U 2 ) 
n U Jm feK* \nfzt ) 

<2 / (m(x) - w) 2 p(x)dx + 2 m 2 + (u>max + W m i n /2)C t SUp |p(/) - Pn (/) | ( ~ Y^ uf ) 
Jm fe> c t ,, n -/c t ,, n V n i^i / 


<^E R 


m 2 


n 2 t 


* j=i 


4f 


(rti Uj) 


+ max{w max + w min /2, 2/wi min }C' t sup |p(/) - Pn(/)| ( - Y^ u 2 

/e/c t ,, n -/c t ,,„u/c t ,,„ \ n 1~l , 

Let S = j —^ ~ ET 1 w 2 ( x i) > — Er 1 and 

4tt?max+3tU m in n ^1=1 v LJ ~ Tl '2=1 l 5 

max{ii; max + uw/2, 2/w min }C t sup |p(/) - p„(/)| < <5 2 /2 


then we have completed the proof. Otherwise, we have 


1 


- J2( Ui - u ( x *)) 2 = -J2 u2 i + ~Y1 u ^ 2 ~~Y1 ^ ——— 5Z 


1 


(i-<5) 2 


(A.9) 


2=1 2=1 2=1 2=1 2=1 
This enables us to prove the theorem in the case of \ Ei*=i u2 ( x *) < tt E”=i u i as follows. 


C t 




J2 r 


i,j =1 


4t' 




n“ 


hi =1 


At’ 


(Ui-Uj) 2 


Ui(Ui - Uj ) 


2 x -v 

= -Y- w(xj))iy t /„(xj) 

n z 


2=1 


= - Y^(w* - u(Xj)) 2 W t / „(Xj) + - Y' u(x.i)(ui - u(Xi))w t ' ,n('X-i) 

n z —' n z —' 

2=1 2=1 

2 71 /l n \ 1/2 / x n 

> - “ u(x i )) 2 Wt', rl (Xj) - 2 - ^2 U 2 (Xi)w t ',n(*i) - ^2{ui - M( x 0) 2 W t ,n( x i) 

71 i=l \ n i=l / \ n i=l 


1/2 


> 


W miE 


^ ' Yz Y X 0) 2(in max + l^ m i n /2) I ^ ' U (Xj) 

»=i \ n i=l y 

1/2 


1/2 


^( Ui - U ( Xi )) 2 


1/2 


i=l 


/ ^ ^ \ / | n ^ 

^ (^minY 2(lC max T lC m i n /2)(5) I ^ ' Uj I I ^ ' ('U'i ^( x i)) 


1/2 


> Wm in(l-<5) 2 (-^1 


(A.10) 
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